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(f)] distinct from the local time lines so as to initiate re-mapping of said start time and 
duration ion each said clip objects represented by said meta-clip [object] objects according to the 
relatiY^positioning of said local time [line] lines and said global time line. 



--REMARKS - - 

In the Office Action, the Examiner objected to the title for failing to clearly describe the 
invention. In response, the Applicant has amended the title as set out above. 



Claims 1 to 1 1 are presently pending in the subject patent application, and stand rejected under 
35 USC 103(a) in view of Hill (US 5 7 930,797) and Hamakawa (Hamakawa and Rekimoto, 
"Object Composition and Playback Models for Handling Multimedia Data", International Media 
Conference, Proceedings of the Conference on Multimedia '93, August 2-6, 1993). In response 
to the Office Action, the Applicant has amended independent claims 1, 4 and 1 1 , as set out 
above. The Applicant now submits that the invention, as defined in claims 1 to 11, patentably 
distinguishes over the cited prior art. The basis of the Applicant's position will be explained in 
the following paragraphs. 



Rejection of Claims 1 to 3 

Independent claim 1 of the subject patent application relates to a method for accessing and 
manipulating time-based data. The claimed method (as amended herein) comprises the steps of: 

(i) selecting a first time-based data source comprising a first data type from a selection of 
available data sources; 

(ii) positioning a first clip object representing said first time-based data source with 
respect to a local time line to define a start time and duration relative to the local time line for 
accessing said first time-based data source; 

(iii) selecting a second time-based data source from said selection of available data 
sources, said second time-based data source being of a different data type than said first time- 
based data source; 
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(iv) positioning a second clip object representing said second time-based data source with 
respect to said local time line to define a start time and duration relative to the local time line for 
accessing said second time-based data source; 

(v) creating at least one meta-clip object representing said local time line and said first 
and second clip objects positioned relative thereto, said at least one meta-clip object being 
positionable with respect to a global time line of an edit, distinct from said local time line, such 
that the start time and duration of each of said first and second clip objects in said at least one 
meta-clip are re-mapped to said global time line upon said at least one meta-clip being positioned 
relative to the global time line; and 

(vi) adding said at least one meta-clip object to said last of available data sources. 

A distinguishing feature of the present invention, as defined in amended claim 1 , is that the clip 
objects comprising the meta-clip object are associated with a time line which is local to each 
meta-clip, but at the same time distinct from the global time line used to define the edit. By 
defining an edit in this manner, the relationship between a group of associated data types can be 
maintained, even if the group of data types is translated or scaled in time. Another distinguishing 
feature of the present invention is that the group of associated data types, so defined, is stored as 
pan of a pool of available data sources, thereby facilitating the development of more complex 
data types, including more complex meta-clips. None of the prior art references cited by the 
Examiner teach or suggest an invention which includes these features. 

Hill fUS 5,930.797) 

Hill teaches a method of representing hierarchical time-based data of an edit to increase the 
efficiency of a rendering process. The method involves representing each edit track of the edit as 
a hierarchy of objects within a media container, creating a minimum set of virtual static graphs 
from the media container and then outputting the graphs to a rendering engine. As the patentee 
discloses at column 6, line 54 to column 7, line 16 of the patent, a media container comprises a 
hierarchy of source objects, effects objects, filter objects and masks arranged in layers, 
constrained such that an object in any given layer cannot receive input from another object in the 
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same layer or from an object in a higher layer. The media container also includes a layer 
multiplexer for each layer for maintaining a record of the active/inactive state of each object in 
the layer at any given time, and the active/inactive state of each object in adjacent lower layer. 

Once the media container is defined, a minimum number of virtual static graphs is created to 
determine the outputs which are required to be applied to the rendering engine at any given time. 
As the patentee discloses at column 9, line 57 to column 10, line 48 of the patent, virtual static 
graphs are created by first identifying transitions (ie the instant when the state of an object 
changes), and then for each duration between adjacent transitions determining whether the 
required output for the duration is a source object or an effect/filter object. If the required output 
is a source objectfs), then a virtual static graph for the duration is created from the source 
object(s). However, if the required output is an effect object or a filter object, the layer 
multiplexer associated with the output object is studied to determine whether the objects which 
provide inputs to the output object are available. If a required input is available, then a virtual 
static graph for the duration is created using the input object. On the other hand, if the required 
input is not available, the layer multiplexer of each subsequently lower layer is examined until 
the required input is found available, and a virtual static graph for the duration is created from the 
intervening objects. 

As will be apparent from the foregoing description, Hill does not teach a method for accessing 
and manipulating time-based data which involves the step of creating one or more meta-clip 
objects by positioning clip objects representing respective time-based data sources relative to a 
local time line, with each meta-clip object being positionable relative to a global edit time line 
which is distinct from the local time lines, as recited in amended claim I of the subject patent 
application. Further, Hill does not teach a method for accessing and manipulating time-based 
data, which involves the step of including the meta-clips, so created, as part of a pool of available 
data sources which can be used for the creation of an edit. Instead, Hill discloses a media 
container which includes either source objects or effect/filter objects all disposed relative to a 
global edit time line, without any of the objects making reference to a local time line. Also, as 
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the patentee discloses at column 6, lines 50 to 53 of the patent, a container is not saved once the 
edit itself is saved. Consequently, the invention recited in amended claim 1 of the subject patent 
application is not anticipated by Hill. 

Hamakawa (Object Composition and Playback Models for Handling Multimedia Data) 
Hamakawa teaches an object-oriented composite model for handling multimedia data. As the 
author describes at pg. 274 of the publication, the composite object comprises a hierarchical 
arrangement of multimedia objects, each having glue attributes (normal, stretch, shrink) in three 
dimensions (two dimensional position, time). In addition to the glue attributes, each multimedia 
object has associated therewith data type information, and location information specifying the 
relative location (in terms of time and space) of the object relative to other multimedia objects. 
Once the objects are composed, their respective absolute locations (in terras of time and space) 
are then calculated, with the time length of each object comprising the composite object being 
determined from the time length of the composite object. If the attribute values of an object is 
subsequently changed, the attribute values of the other objects must be recalculated, in 
accordance with the object hierarchy. 

As will be apparent from the foregoing description, Hamakawa does not teach a method for 
accessing and manipulating time-based data, which involves the step of creating one or more 
meta-clip objects by positioning clip objects representing respective time-based data sources 
relative to a local time line, with each meta-clip object being positionable relative to a global edit 
time line which is distinct from the local time lines, as recited in amended claim I of the subject 
patent application. Instead, Hamakawa discloses an object-oriented composite model which 
differs from the conventional time line model in that users do not specify the absolute location of 
the objects. As the author discloses at page 274, rather than specifying the location of each 
object relative to a global edit time line, the users specify the location of each object relative to 
the other objects in the object hierarchy. Although Hamakawa discloses that once objects are 
defined, their absolute locations (in terms of time and space) are calculated, Hamakawa does not 
make use of a local time line to locate one or more associated clip objects (ie. meta-clip objects) 
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and a global time line to locate one or more meta-clip objects. Consequently, the invention 
recited in amended claim 1 of the subject patent application is not anticipated by Hamakawa. 

The Applicant also submits that the invention recited in amended claim 1 of the subject patent 
application is not obvious in light of Hill and Hamakawa. The initial basis for the Applicants 
position is that for a modified prior art reference to properly form the basis of a prima facie 
obviousness rejection, there must be some suggestion in the prior art for the modification. As the 
Applicant discusses below, the required suggestion is lacking. 

As discussed above, amended claim 1 relates to a method for accessing and manipulating time- 
based data, which involves the steps of: 

selecting at least two time-based data sources from a selection of available data sources, 
and positioning clip objects representing the respective data sources with respect to a local time 
line to define a start time and duration relative to the local time line for accessing each data 
source; 

creating at least one meta-clip object representing the local time line and the clip objects 
positioned relative thereto, with the meta-clip object being positionable with respect to a global 
time line of an edit, distinct from said local time line, such that the start time and duration of each 
clip object in each meta-clip are re-mapped to the global time line when the raeta-clips are 
positioned relative to the global time line; and 

adding each meta-clip object to the list of available data sources. 

In contrast, although Hill also relates to a method for representing and accessing time-based data 
which involves the steps of selecting time-based data sources from a selection of available data 
sources, and positioning within a media container a hierarchy of objects representing the selected 
data sources with respect to a time line to define a start time and duration relative to the time line 
for accessing each data source, Hill fails to disclose the step of creating meta-clip objects each 
comprising a collection of clip objects disposed relative to a local time line, with each metan:lip 
object being positionable relative to a global edit time line distinct from the local time lines, as 
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recited in amended claim 1 of the subject patent application. The Applicant notes that the 
Examiner stated that Hamakawa disclosed the use of a composite object comprising multimedia 
objects, each with its own relative time line, temporally re-mapped with respect to a global time 
line. However, this statement is incorrect As the Applicant discussed above, at page 274 of the 
publication Hamakawa disclosed that, rather than specifying the location of each object relative 
to a time line, the users only specify the location of each object relative to the other objects in the 
object hierarchy . Although Hamakawa discloses that once objects are defined, their absolute 
locations (in terms of time and space) are calculated, Hamakawa does not make use of a local 
time line to locate one or more associated clip objects of a group of objects and a global time line 
to locate one or more groups of objects, as employed by the invention recited in amended claim 1 
of the subject patent application. Instead, Hamakawa makes use of the relative locations in time 
and space between objects to remap the objects to a common global time line. Consequently, the 
written disclosure of Hamakawa cannot be used as basis to conclude that a person of ordinary 
skill would be motivated to modify the teaching of Hill to make use of both respective local time 
lines for objects comprising composite objects and a global time line for the composite objects. 

The Applicant also notes that the Examiner referred to Figure 4 of Hamakawa as an example of 
temporal re-mapping of objects from their own time line to a global time line. However, as 
Hamakawa explained at page 274 of the publication, Figure 4 merely depicts a mechanism for 
determining the absolute location (time and space) of a composite object comprising a number of 
multimedia objects. Although Figure 4 does depict a time line, the depicted time line is the 
absolute (global) time line of the edit Figure 4 does not depict one or more clip objects of a 
group of objects all associated with a common local time line, and one or more groups of objects 
all associated with a global time line for the edit, as required by the invention recited in amended 
claim I. Consequently, the drawings in Hamakawa cannot be used as basis to conclude that a 
person of ordinary skill would be motivated to modify the teaching of Hill to make use of both 
respective local time lines for objects comprising composite objects and a global time line for the 
composite objects. 
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The Applicant does note that the Examiner concluded that a person of ordinary skill would be 
motivated to apply the teaching of Hamakawa to the Teaching of Hill due to "Hamakawa's taught 
advantage of automatic temporal re-mapping of time lines within groupings of multimedia 
objects, providing increased convenience (due to the elimination for precise time line locations)". 
However, as the Applicant explained above, Hamakawa does not make use of time lines local to 
a group of multimedia objects and a separate global time line for the edit, but instead simply 
determines the locations in time and space of objects relative to other objects (absent any local 
time line) and then remaps the relative locations of the objects to a global time line. 
Consequently, the requisite suggestion in the prior art for the modification of Hill to make use of 
a collection of clip objects disposed relative to a local time line, with each collection of clip 
objects being positionable relative to a global edit time line distinct from the local time lines is 
entirely lacking. 

The Applicant also points out that at column 2, lines 26 to 41 and at column 5, lines 45 to 48 of 
the patent, Hill disclosed that since a NLE edit can include many source data clips, effects 
objects and filter objects, the computational analysis necessary to render the edit can be 
computationally expensive, particularly if the edit includes redundant interconnections and/or 
data elements. To reduce the computational overhead required for the rendering of an edit, Hill 
required that a minimum set of virtual static graphs be created for output to the rendering system, , 
However, if the source, filter or effect objects in any layer of the media container taught by Hill 
were replaced by composite objects, such as the composite model taught by Hamakawa, it would 
be possible for the number of virtual static graphs to be other than minimum since two composite 
objects in any given layer of the media container could include some of the same 
interconnections and/or some of the same source, effect and/or filter objects, thereby passing to 
the rendering system an edit comprising possibly redundant interconnections and/or data 
elements. Although this problem might be solved by "'expanding 7 ' each composite object into its 
constituent multimedia objects prior to inclusion in the media container, the requisite expansion 
of each composite object would be computationally expensive, particularly since Hill does not 
save media containers after the edit itself is saved. 
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Further, the composite model taught by Hamakawa is not particularly conducive to a 
computationally efficient dynamically editable rendering system. As discussed above, 
Hamakawa discloses a static composite object model which requires that users specify the 
locations in time and space of each multimedia object comprising the composite object relative 
to the other multimedia objects comprising the composite object (absent any local time line), 
prior to remapping the relative locations to a global time line. Although such a model might be 
suitable for defining complex static objects, it is not suitable for applications requiring the 
composite models to be dynamically modified (except for variations in object's glue attributes) 
since if the user repositioned one multimedia object in the composite object, the user would also 
have to reposition all the other multimedia objects in the composite object whose locations were 
originally defined with respect to the repositioned multimedia object. Since edits generally 
include a large number of clips, the requirement to reposition multiple associated clips every 
time a change is made to a clip could make editing very time consuming. Accordingly, not only 
does the prior art fail to suggest modifying Hill to make use of a collection of clip objects 
disposed relative to a local time line, with each collection of clip objects being positionable 
relative to a global edit time line distinct from the local time lines, but the prior art teaches away 
from the notion of modifying the teaching of Hill to incorporate the teaching of Hamakawa. 

Summarizing the foregoing, the prior art fails to disclose a method for accessing and 
manipulating time-based data which involves the step of creating one or more meta-clip objects 
by positioning clip objects representing respective time-based data sources relative to a local 
time line, with each meta-clip object being positionable relative to a global edit time line which 
is distinct from the local time lines, as recited in amended claim 1 of the subject patent 
application. Although Hill relates to a method for representing and accessing time-based data 
which involves the steps of selecting time-based data sources from a selection of available data 
sources, and positioning within a media container a hierarchy of objects representing the selected 
data sources with respect to a time line, the prior an fails to suggest modifying Hill to make use 
of a collection of clip objects disposed relative to a local time line, with each collection of clip 
objects being positionable relative to a global edit time line distinct from the local time lines. 
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Further, the prior an actually teaches away from modifying Hill to make use of predefined 
composite objects, such as the composite model taught by Hamakawa, Therefore, Hill cannot be 
properly used as the basis of a prima facie obviousness rejection of the invention recited in 
amended claim 1 of the subject patent application. Accordingly, The Applicant submits that the 
invention recited in amended claim 1 is not obvious in view of the cited references. 

As claims 2 to 3 depend from independent claim 1, the foregoing arguments apply equally to 
claims 2 to 3. Accordingly, the Applicant respectfully requests that the Examiner's rejection of 
claims 1 to 3 be withdrawn. 

Rejection of Claims 4 to 10 

Independent claim 4 of the subject patent application relates to a method of defining an edit 
comprising time-based data disposed relative to a global time line. The claimed method (as 
amended herein) comprises the steps of: 

(i) creating at least one meta-clip object each comprising a respective local time line 
distinct from the global time line, a first clip object representing a first time-based data source 
selected from a list of available data sources, and a second clip object representing a second time- 
based data source selected from the list of available data sources, the second data source being of 
a different data type than the first data source, the first and second clip objects being positioned 
relative to the local time line to define a respective start time and duration relative to the local 
time line for accessing each said selected data source; 

(ii) adding said at least one meta-clip object to said list of available data sources; 

(iii) selecting at least one of the meta-clip objects from said list of available data sources 
and positioning said at least one selected meta-clip object with respect to the global time line; 
and 

(iv) re-mapping to the global time line the start time and duration of the clip objects 
comprising each said selected meta-clip object in accordance with the position of each said 
selected meta-clip object relative to the global time line. 
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The Examiner wiU note that claim 4 has been amended to conform substantially to amended 
claim 1. In particular, amended claim 4 recites the steps of: 

creating at least one meta-clip object each comprising a respective local time line, a first 
clip object representing a first time-based data source selected from a list of available data 
sources, and a second clip object representing a second time-based data source selected from the 
list of available data sources, with the second data source being of a different data type than the 
first data source; 

adding the meta-clip object to the list of available data sources; and 
selecting at least one of the meta-clip objects from the list of available data sources and 
positioning the meta-clip relative to a global (edit) time line. 

In view of the substantial similarity between amended claim 4 and amended claim 1, the 
foregoing arguments apply equally to the invention defined in amended claim 4. Accordingly, 
the Applicant submits that the invention recited in amended claim 4 is neither anticipated nor 
obviousness in view of the cited references. 

As claims 5 to 10 depend from independent claim 4, the foregoing arguments apply equally to 
claims 5 to 10. Accordingly, the Applicant respectfully requests that the Examiner's rejection of 
claims 4 to 10 be withdrawn. 

Rejection nf Claim 11 

Independent claim 1 1 of the subject patent application relates to a non-linear editing system for 
creating an edit by accessing and manipulating time-based data. The claimed system (as 
amended herein) comprises: 

a storage device to store time-based data sources; 

a computer operatively connected to said storage device to access said time-base data 
sources stored therein; 

at least one output device to display to a user a graphical user interface of an NLE 
program executed by said computer and to output the result of said edit to said user; and 
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at least one user input device to receive input for said NLE program from a user, said 
input being configured to: 

(a) create with the computer at least one meta-clip object each comprising a respective 
local time line, a first object representing a first one of the stored data sources, a second object 
representing a second one of the stored data sources, the second data source being of a different 
data type than tbefirst data source, the clip objects being positioned relative to the local time line 
to define a respective start time and duration relative to the local time line for accessing each said 
data source; 

(b) select with the computer at least one of the meta-clip objects; and 

(c) define with the computer the positioning of each said selected meta-clip object 
relative to a global time line distinct from the local time lines so as to initiate re-mapping of said 
start time and duration of each said clip objects represented by said meta-clip objects according 
to the relative positioning of said local time lines and said global time line. 

The Examiner will note that claim 1 1 has been amended to conform substantially to amended 
claim 1 . In particular, amended claim 1 1 recites an input device which is configured to: 

create at least one meta-clip object each comprising a respective local time line, a first 
object representing a first one of the stored data sources, a second object representing a second 
one of the stored data sources, the second data source being of a different data type than thefirst 
data source; 

select at least one of the meta-clip objects; and 

define the positioning of each selected meta-clip object relative to a global time line 
distinct from the local time lines so as to initiate re-mapping of the start time and duration of the 
clip objects represented by the meta-clip objects according to the relative positioning of the local 
time lines and the global time line. 

In view of the substantial similarity between amended claim 1 1 and amended claim 1, the 
foregoing arguments apply equally to the invention defined in amended claim 1 1 . Accordingly, 
the Applicant submits that the invention recited in amended claim 1 1 is neither anticipated nor 
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obviousness in view of the cited references. Accordingly, the Applicant respectfully requests 
that the Examiner's rejection of claim 1 1 be withdrawn. 

Favourable reconsideration of the subject patent application is respectfully requested. 

If any additional fees are required by any of the foregoing amendments or submissions, 
permission is hereby granted to debit our deposit account number 07-1750. 

If the Examiner wishes to discuss any aspect of this amendment, please contact the Applicant's 
patent agent, Mr. Robert Graham, at (416) 862-4425. 



Respectfully submitted, 




Robert J. Graham 



Reg No. 43,430 



INTELLECTUAL PROPA26 5364.1 
December 1,2000 
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